AD-A174  267 


Final  Report:  Adaptive  Assessment  of  Spatial  Abilities 


Isaac  I.  Be jar 
Edu:ational  Testing  Service 
Princeton,  NJ  08541 

/ 


June  1986 


DTIC 


FLECTE 
NOV  1  8  1988 


Approved  for  public  release;  distribution  unlimited. 
Reproduction  in  whole  or  in  part  is  permitted  for  any  purpose  of 
the  United  States  Government. 


This  research  was  sponsored  by  the  Personnel  and  Training 
Research  Programs,  Psychological  Sciences  Division,  Office  of 
Naval  Research,  under  Contract  No.  N00014-83-C-0761,  Contract 
Authority  Identification  No.  NR  150  531 


er 


86  U  18  006 


mammmmmmmrnmmmmmmmmmmmsm 


SECURITY  CLASSIFICATION  OF  THIS  PAGE 


S' 


REPORT  DOCUMENTATION  PAGE 

la.  REPORT  SECURITY  CLASSIFICATION 

Unclassified 

1b.  RESTRICTIVE  MARKINGS 

2a.  SECURITY  CLASSIFICATION  AUTHORITY 

3  DISTRIBUTION /AVAILABILITY  OF  REPORT 

Approved  for  public  release; 

2b.  DECLASSIFICATION  /  DOWNGRADING  SCHEDULE 

distribution  unlimited 

4  PERFORMING  ORGANIZATION  REPORT  NUMBER(S) 

S.  MONITORING  ORGANIZATION  REPORT  NUMBER(S) 

63.  NAME  OF  PERFORMING  ORGANIZATION 

Educational  Testing  Service 


6c.  ADORESS  (City,  State,  and  ZIP  Code) 
Princeton,  NJ  08541 


6b.  OFFICE  SYMBOL 
(If  applicable) 


7b  ADDRESS  (City,  State,  and  ZIP  Code) 
Arlington,  VA  22217-5000 


83.  NAME  OF  FUNDING /SPONSORING 
ORGANIZATION 


8b,  OFFICE  SYMBOL 
(If  applicable) 


8c  ADDRESS  (City,  State,  and  ZIP  Code) 

Office  of  Naval  Research  (Code  1142PT) 
800  North  Quincy  Street 
Arlington.  VA  22217-500 


11  TITLE  (Include  Security  Classification) 

Final  Report:  Adaptive  Assessment  of  Spatial  Abilities 


12  PERSONAL  author<S) 

Isaac  I.  Be jar 


9.  PROCUREMENT  INSTRUMENT  IDENTIFICATION  NUMBER 

N00014-83-C-0761 


10.  SOURCE  OF  FUNDING  NUMBERS 


PROGRAM 
ELEMENT  NO 

PROJECT 

NO 

!■  VII  In  1 1  WM 

min  i 

RR0420401 


13a  type  of  report 
Final 


16  SUPPLEMENTARY  NOTATION 


13b  TIME  COVERED 
FROM  10/83  TO  3 


14.  DATE  OF  REPORT  (Year,  Month,  Day)  Il5  PAGE  COUNT 
86  June 


COSATl  COOES 


GROUP 


09 


SUB-GROUP 


18  SUBJECT  TERMS  (Continue  on  reverse  if  necessary  and  identify  by  block  number) 

Spatial  ability  item  response  theory 
cognitive  psychology  computerized  testing 


19  ABSTRACT  ( Continue  on  reverse  if  necessary  and  identify  by  block  number) 

This  report  summarizes  the  results  of  an  18-month  contract  entitled 
Adaptive  Assessment  of  Spatial  Ability.  The  project  was  focused  on  the 
psychometric  and  technological  feasibility  of  adaptive  testing  systems  of 
a  procedural  as  opposed  to  declarative  nature.  That  is,  adaptive  testing 
systems  where  items  ace  generated  as  needed  rather  than  explicitly  retrieved 
from  a  database.  To  Investigate  the  feasibility  of  such  an  approach  to 
adaptive  testing  data  was  collected  from  high  school  students  on  two  types 
of  spatial  items,  three-dimensional  cubes  and  hidden  figure  items.  The 
analysis  of  the  three-dimensional  cube-'  focused  on  the  fit  of  the  simplest 
possible  item  response  model  capable  of  modeling  response  time;  the  analysis 
of  the  hidden  figure  item  focused  on  the  feasibility  of  generating  item  from 
an  algorithm  in  such  a  way  that  the  psychometric  characteristics  of  the 
generated  items  were  predictable.  The  results  for  the  three-dimensional  cube 


20  DISTRIBUTION /AVAILABILITY  OF  ABSTRACT 

BuNCLASSIFIED/UNLIMITED  □  SAME  AS  RPT  □  OTIC  USERS 

21.  ABSTRACT  SECURITY  CLASSIFICATION 

Unclassified 

22 a  NAME  OF  RESPONSIBLE  INDIVIDUAL 

Dr.  Charles  Davis 

22b  TELEPHONE  (Include  Area  Code) 

(202)  696-4046 

22c  OFFICE  SYMBOL 

0NR  1142PT 

DO  FORM  1473, 84  mar 


83  APR  edition  may  be  used  until  exhausted 
All  other  editions  are  obsolete 


SECURITY  CLASSIFICATION  OF  THIS  PAGE 


Block  19.  (Continued) 


'V items  suggested  that  angular  disparity  can  be  used  effectively  to 
control  the  difficulty  of  true  items  but  this  was  not  the  case  for 
false  items.  That  is,  true  and  false  items  appear  to  measure 
different  aspects  of  performance  and  as  a  result  a  multidimensional 
item  response  model  may  be  necessary  to  fully  account  for  performance 
on  even  fairly  simple  spatial  items  such  as  three-dimensional  cubes.  * 
The  analysis  of  the  hidden  figure  items  showed  that  an  item  generation 
algorithm  can  be  formulated  to  produce  items  of  similar  psychometric 
characteristics.  The  practical  and  theoretical  implication  of  the 
results  are  discussed. 


Abstract 
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As  the  title  of  this  project  suggests,  the  aim  of  this  research  is 
to  study  the  feasibility  and  requirements  of  adaptive  testing  for  spatial 
ability.  However,  although  the  content  of  the  research  has  been  spatial 
abilities,  the  goal  is  in  fact  broader,  namely  to  develop  a  methodology 
for  what  might  be  called  second-generation  adaptive  testing  that  will  be 
applicable  not  only  to  spatial  but  to  other  abilities  as  well. 

First-generation  adaptive  testing  methodology  is  well  known  and  can 
be  summarized  as  follows:  Given  a  pool  of  items  calibrated  on  a  common 
scale,  choose  the  set  of  items  that  is  maximally  informative  for  a  given 
examinee.  This  methodology  has  now  reached  the  point  where  it  is  a 
marketable  product,  and  while  there  may  still  exist  a  need  to  do  research 
on  refinements  of  the  methodology,  the  basic  structure  of  the  paradigm  is 
well  set. 

A  characteristic  of  first-generation  adaptive  testing  is  its  declar¬ 
ative  nature.  That  is,  each  item  in  the  pool  must  be  stored  explicitly 
in  a  database  along  with  its  psychometric  parameters  with  respect  to  some 
item  response  model.  A  natural  elaboration  of  this  approach  was  investi¬ 
gated  in  this  project.  That  is,  instead  of  our  explicitly  enumerating 
all  the  items,  we  investigated  the  idea  of  constructing  algorithms  that 
generate  the  items  with  control  of  their  psychometric  characteristics. 
Rather  than  calibrating  specific  items,  we  calibrated  the  procedures  that 
generate  the  items.  In  short,  the  elaboration  moves  from  a  declarative 
approach  to  a  procedural  one. 

Clearly,  procedural  adaptive  testing  involves  more  than  psycho¬ 
metrics,  since  the  encoding  of  items  into  procedures  requires  very 
specific  knowledge  about  the  determinants  of  item  performance.  It  is 
precisely  this  requirement  that  offers  some  hope  of  inproving  the 
validational  status  of  scores  from  an  adaptive  testing  procedure.  The 
current  approach  to  adaptive  testing  improvement  in  validity  is  limited 
to  the  improvement  accruing  from  more  precise  measurement.  There  is  hope 
that  the  next  generation  in  adaptive  testing  will  improve  the  valida¬ 
tional  status  of  test-score  interpretations  by  continually  submitting  to 
testing  the  theory  of  item  performance  embedded  in  the  item-generation 
algorithm.  As  a  result  of  that  continual  challenge,  the  theory  will 
either  be  confirmed  or  revised,  and  it  is  very  likely  that  in  that 
process  we  will  learn  much  about  the  psychological  underpinnings  of 
performance  on  the  test. 

The  calibration  of  a  procedure  consists  of  item  linking  those 
determinants  of  performance  to  a  psychometric  scale.  The  details  of  how 
this  is  done  vary  with  the  item  type.  In  this  project,  we  experimented 
with  a  three-dimensional  mental  rotation  item  and  a  hidden-figure  item 
type. 
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The  Psychometrics  of  Three-dimensional  Mental  Rotation 

An  example  of  this  item  type  is  shown  in  Figure  1.  This  item  type 
was  chosen  because  there  exists  a  large  body  of  literature  (cf., 
Corballis,  1982)  establishing  that  an  angular  disparity  between  the  two 
figures  largely  determines  performance.  Moreover,  it  appears  that  there 
are  fairly  stable  and  consistent  gender  differences  in  performance  on 
mental-rotation  tasks  (Linn  and  Petersen,  1985). 

The  approach  taken  was  to  examine  the  simplest  possible  psychometric 
model  of  an  80-item  test  based  on  figures  such  as  those  in  Figure  1. 
(There  were  eight  basic  items  presented  at  five  angles  in  their  true  and 
false  version.)  The  items  were  presented  at  angular  disparities  of  20, 
60,  100,  140,  and  180  in  order  to  establish  the  relationship  between 
angular  disparity  and  difficulty.  The  simplest  model  that  can  be  fitted 
to  these  data  makes  the  following  predictions: 

-  The  relationship  between  difficulty  and  angular 
disparity  is  linear. 

-  The  slope  of  that  relationship  is  constant  at 
different  response  times. 

-  The  intercept  of  the  relationship  is  solely  a 
function  of  response  time. 

This  model  is  an  extension  of  the  dichotomous  item-response  model  to 
the  case  in  which  the  response  is  response  time  (see  Samejima,  1973). 
Thus,  to  score  an  examinee,  we  simply  note  the  response  time  to  an  item 
with  a  certain  angular  disparity.  Together,  the  angular  disparity  and 
response  time  determine  the  corresponding  difficulty,  and  they  allow  us 
to  obtain  an  ability  score  for  this  examinee. 

Figure  2  shows  the  result  of  a  calibration  for  a  typical  item  based 
on  the  responses  of  nearly  200  high  school  students.  As  can  be  seen, 
there  are  some  departures  from  the  predictions  although,  in  general,  the 
fit  for  this  item  is  good.  The  major  deviation  from  linearity  occurred 
at  100  degrees.  Also,  beyond  5  seconds,  a  tendency  towards  a  quadratic 
relationship  between  difficulty  and  angular  disparity  emerges,  a  situa¬ 
tion  which  suggests  that  beyond  a  certain  moment  in  time  different 
strategies  come  into  play. 

The  results  for  the  false  items  are  quite  different,  in  that  angular 
disparity  does  not  seem  to  control  performance  as  it  does  for  the  true 
items.  That  is,  the  false  items  seem  to  tap  the  decision  aspect  of 
performance,  while  the  true  items  are  tapping  the  mental  rotation  aspect. 
Figure  3  shows  the  corresponding  data. 

The  results  of  this  study  are  presented  in  more  detail  in  The 
Psychometrics  of  Mental  Rotation  (RR-86-19).  It  is  concluded  that  in 
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Fioure  1 

Sample  True  and  False  Three-dimensional  Rotation  Items 


Figure  2 

Relationship  Between  Psychometric  Difficulty  and  Angular  Disparity 
After  3,  4,  5,  6  and  7  Seconds  for  True  Version  of  Item  El 


Figure  3 

Relationship  Between  Psychometric  Difficulty  and  Angular  Disparity 
After  3,  4,  5,  6  and  7  Seconds  for  False  Version  of  Item  El 


practical  applications,  the  appropriate  psychometric  model  for  this  item 
type  is  a  two-dimensional  one.  However,  in  a  computerized  testing 
environment,  it  may  be  unnecessary  to  embellish  the  psychometric  model  to 
account  for  curvilinear  relationships  between  angular  disparity  and 
difficulty.  Instead,  in  the  tailoring  of  the  test  we  chose  items  for  an 
individual  in  such  a  way  that  a  response  is  given  within,  say,  5  seconds. 
Such  a  tailoring  strategy  may  have  other  benefits  as  well. 

Hidden  Figure  Items 

Unlike  the  mental-rotation  items,  for  which  the  determinants  of 
performance  are  fairly  well  known,  very  little  is  known  about  the  deter¬ 
minants  of  performance  in  hidden-figure  items.  Therefore,  our  first  task 
was  to  discover  a  psychometrically  useful  representation  of  the  item. 
There  were  two  important  constraints  on  that  representation.  One  was 
that  it  should  provide  a  description  of  the  item  that  captures  the 
"psychometric  essence"  of  the  items.  Ideally,  that  representation  should 
be  psychologically  motivated,  that  is,  motivated  by  previous  research  on 
the  processes  and  mental  models  that  account  for  performance  on  this  type 
of  cognitive  task.  Unfortunately,  for  the  hidden-figure  item,  it  was  not 
possible  to  locate  the  relevant  research.  In  addition,  the  representa¬ 
tion  should  lend  itself  to  generating  items  that  had  the  same  underlying 
representation  but  a  different  visual  realization.  For  convenience,  we 
call  the  items  generated  in  this  fashion  clones .  Figure  4  shows  a  pair 
of  clones. 

The  chosen  representation  is  a  matrix  consisting  of  counts  indi¬ 
cating  how  close  the  target  figure  appears  at  each  possible  position  in 
the  larger  pattern  and  was  based  on  the  Hough  transform  (Fayhew  and 
Frisby,  1984),  an  artificial  intelligence  technique  used  in  object 
recognition.  We  tested  the  psychometric  validity  of  this  representation 
by  implementing  a  computer  program  capable  of  generating  psychometric 
clones  and  then  by  comparing  their  psychometric  characteristics  on  the 
basis  of  responses  from  high  school  students. 

The  item  generation  algorithm  takes  the  matrix  of  counts  together 
with  a  small  pattern  and  tries  to  create  a  large  pattern  that  matches  the 
matrix.  The  generation  process  is  simplified  by  the  fact  that  patterns 
only  contain  horizontal,  vertical,  and  45  degree  lines  between  nodes. 

The  basic  idea  is  to  start  with  a  large  pattern  including  all  the 
possible  lines  and  remove  lines  until  the  matching  algorithm  produces  a 
matrix  that  equals  the  input  matrix. 

The  results  demonstrated  that  the  clones  behaved  as  such  in  terms  of 
their  difficulty  as  well  as  distribution  of  response  times.  Figure  5 
shows  the  relationship  between  the  logit  for  proportion  correct  and  for 
pairs  of  clones  as  well  as  the  corresponding  mean  response  time.  Figure 
6  shows  the  cumulative  response  times  for  two  clones.  It  can  be  seen 
they  are  very  similar,  and  this  was  true  for  the  other  items  as  well. 

The  results  of  this  experiment  appear  in  more  detail  in  Analysis  and 
Generation  of  Hidden  Figure  Items;  A  Cognitive  Approach  to  Psychometric 
Modeling  (RR-86-20). 
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Fioiire  5 

Relationship  Between  Accuracy  and  Latency  for  Hidden  Fioure  Clones 


RELATIONSHIPS  BETWEEN  DIFFICULTY 

FOR  CLONES  ft  AND  6 


RELATIONSHIPS  BETWEEN  RESPONSE  LATENCY 

FOR  CLONES  A  AND  E 
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Summary 

The  choice  of  item  types  in  this  study  was  not  accidental:  they 
were  chosen  to  maximize  the  chance  of  a  positive  demonstration  of  what  we 
have  called  "procedural  adaptive  testing."  The  essential  characteristic 
of  procedural  adaptive  testing  is  that,  unlike  "conventional"  adaptive 
testing,  all  the  items  and  their  associated  item  parameter  estimates  need 
not  be  stored  ahead  of  time  in  a  database.  Instead,  through  a  design 
incorporating  the  major  determinants  of  performance  on  that  item,  data 
are  collected  to  determine  the  relationship  between  design  and  psycho¬ 
metric  parameters.  This  simple  distinction,  however,  has  important 
ramifications. 

At  a  practical  level,  procedural  adaptive  testing  is  likely  to  be 
more  economical  since  it  avoids  the  need  to  calibrate  a  large  number  of 
items.  This  economy  may  prove  advantageous  even  in  paper-and-pencil 
tests  by  facilitating  the  creation  of  a  priori  parallel  forms  and,  in 
general,  by  better  controlling  the  psychometric  characteristics  of  the 
items  that  are  placed  on  the  test.  (In  fact,  the  item-generation  program 
developed  for  the  hidden-figure  item  has  been  used  in  the  development  of 
a  Navy  pilot  test.) 

However,  the  most  important  implication  of  procedural  adaptive 
testing  may  not  be  its  practical  value  but  the  constraint  that  it  imposes 
on  the  psychometrician.  It  is  no  longer  sufficient  to  gather,  calibrate, 
and  link  items — as  if  these  tasks  were  not  demanding  enough.  To  imple¬ 
ment  a  procedural  adaptive  test,  it  is  also  necessary  to  have  a  theory  of 
item  performance  at  a  level  of  specificity  that  new  items  can  be  produced 
on-line  and  under  computer  control.  These  are  not  trivial  requirements, 
especially  in  verbal  domains.  Thus,  in  attempting  to  fulfill  this 
requirement  it  will  be  necessary  to  gather  documentation  of  psychological 
research  related  to  performance  on  the  item  type  in  question,  and  if  that 
knowledge  is  not  yet  available,  go  ahead  and  obtain  it.  This  process 
will  inevitably  lead  to  a  better  understanding  of  test  scores. 

Conclusions 

Psychologists,  from  psychometric  and  cognitive  perspectives,  have 
been  interested  in  spatial  ability  for  some  time.  Psychometricians 
should  clearly  be  credited  with  the  discovery  and  initial  study  of 
"spatial  abilities."  But  it  is  equally  clear  that  cognitive  psychol¬ 
ogists  deserve  credit  for  the  understanding  we  have  today  about  the 
nature  of  those  abilities.  Having  a  better  understanding,  however,  does 
not  mean  that  we  are  more  certain  about  how  to  measure  spatial  abilities. 
Just  and  Carpenter  (1985),  for  example,  concluded  that  "item  and  test 
difficulty  may  be  major  determinants  of  what  strategies  and  processes 
will  be  evoked  in  a  task."  By  suggesting  that  item  and  test  difficulty 
are  causes,  rather  than  the  result  of  those  strategies  and  processes, 
they  seem  to  suggest  that  psychometric  and  psychological  models  are 
concerned  with  different  phenomena.  The  alternative  view  is  that  not 
only  are  both  models  attempting  to  explain  different  manifestations  of 
the  same  phenomena,  but  in  addition  the  parameters  of  the  psychometric 
model  ought  to  be  explainable  by  the  psychological  theory. 


Adopting  this  view  creates  the  potential  for  measurement  instruments 
that  are  both  theoretically  and  psychometrically  sound.  Although  this 
project  focused  from  the  start  on  the  development  of  more  advanced  adap¬ 
tive  tests,  it  seems  that  even  if  this  had  not  been  the  case  the  conclu¬ 
sion  about  the  need  for  adaptive  testing  would  have  been  inescapable. 

If,  as  Just  and  Carpenter  suggest,  different  strategies  are  invoked  by 
items  of  a  certain  difficulty  level,  then  it  appears  that  a  valuable 
contribution  of  adaptive  testing  is  its  preventing  the  use  of  different 
strategies  by  controlling  the  difficulty  of  items  presented  to  the 
examinee.  The  three-dimensional  rotation  data  collected  as  part  of  this 
project  suggest  that  different  strategies  may  emerge  if  an  examinee  has 
not  made  a  decision  after  five  seconds.  In  an  adaptive  test  it  would  be 
relatively  simple  to  select  items  in  such  a  way  that  the  response  would 
be  given  within,  say,  five  seconds.  This  motivation  for  tailoring  does 
not  negate  the  valuable  information  that  may  lie  in  the  ability  to  choose 
different  strategies.  Rather,  through  better  control  of  what  a  given 
test  measures,  we  are  likely  to  improve  the  precision  and  validity  of 
test  outcomes.  Indeed,  we  may  be  able  to  detect  with  more  certainty  the 
presence  of  alternative  strategies  by  being  able  to  identify  respondents 
that  depart  from  an  expected  pattern  of  performance. 
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Personnel  Analysis  Division, 
AF/MPXA 

5C360,  The  Pentagon 
Washington,  DC  20330 

Air  Force  Human  Resources  Lab 
AFHRL/MPD 

Brooks  AFB,  TX  78235 

Dr.  Earl  A.  Alluisi 
HQ,  AFHRL  (AFSC) 

Brooks  AFB,  TX  78235 

Dr.  Erling  B.  Andersen 
Department  of  Statistics 
Studiestraede  6 
1455  Copenhagen 
DENMARK 


Dr.  Bruce  Bloxom 
Administrative  Sciences 
Code  54B1 

Navy  Postgraduate  School 
Monterey,  CA  93943-5100 

Dr.  R.  Darrell  Bock 
University  of  Chicago 
N0RC 

6030  South  Ellis 
Chicago,  IL  60637 

Cdt.  Arnold  Bohrer 
Sectie  Psychologisch  Onderzoek 
Rekruterings-En  Selectiecentrum 
Kwartier  Koningen  Astrid 
Brui jnstraat 
1120  Brussels,  BELGIUM 


Dr.  Phipps  Arabie 
University  of  Illinois 
Department  of  Psychology 
603  E.  Daniel  St. 

Champaign,  IL  61820 

Technical  Director,  ARI 
5001  Eisenhower  Avenue 
Alexandria,  VA  22333 

Dr.  Eva  L.  Baker 
UCLA  Center  for  the  Study 
of  Evaluation 
145  Moore  Hall 
University  of  California 
Los  Angeles,  CA  90024 

Dr.  Isaac  Be jar 
Educational  Testing  Service 
Princeton,  NJ  08450 

Dr.  Menucha  Birenbaum 

School  of  Education 

Tel  Aviv  University 

Tel  Aviv,  Ramat  Aviv  69978 

ISRAEL 

Dr.  Arthur  S.  Blaiwes 
Code  N711 

Naval  Training  Systems  Center 
Orlando,  FL  32813 


Dr .  Robert  Breaux 
Code  N-095R 

Naval  Training  Systems  Center 
Orlando,  FL  32813 

Dr.  Robert  Brennan 
American  College  Testing 
Programs 
P.  0.  ox  168 
Iowa  City,  IA  52243 

Dr.  Patricia  A.  Butler 
0ERI 

555  New  Jersey  Ave.,  NW 
Washington,  DC  20208 

Mr.  James  W.  Carey 
Commandant  (G-PTE) 

U.S.  Coast  Guard 

2100  Second  Street,  S.W. 

Washington,  DC  20593 

Dr.  James  Carlson 
American  College  Testing 
Program 
P.0,  Box  168 
Iowa  City,  IA  52243 

Dr.  John  B.  Carroll 
409  Elliott  Rd. 

Chapel  Hill,  NC  27514 
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Dr.  Robert  Carroll 
OP  01B7 

Washington,  DC  20370 


Dr.  Norman  Cliff 
Department  of  Psychology 
Univ.  of  So.  California 
University  Park 
Los  Angeles,  CA  90007 

Director, 

Manpower  Support  and 
Readiness  Program 
Center  for  Naval  Analysis 
2000  North  Beauregard  Street 
Alexandria,  VA  22311 

Dr.  Stanley  Collyer 
Office  of  Naval  Technology 
Code  222 

800  N.  Quincy  Street 
Arlington,  VA  22 217-5000 

Dr .  Hans  Crombag 
University  of  Leyden 
Education  Research  Center 
Boerhaavelaan  2 
233*1  EN  Leyden 
The  NETHERLANDS 

CTB/McGraw-Hill  Library 
2500  Garden  Road 
Monterey,  CA  93940 

Dr.  Dattprasad  Divgi 
Center  for  Naval  Analysis 
4*401  Ford  Avenue 
P.0.  Box  16268 
Alexandria,  VA  22302-0268 

Dr.  Hei-Ki  Dong 
Ball  Foundation 
800  Roosevelt  Road 
Building  C,  Suite  206 
Glen  Ellyn,  IL  60137 

Defense  Technical 
Information  Center 
Cameron  Station,  Bldg  5 
Alexandria,  VA  22314 
Attn:  TC 
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Dr.  Stephen  Dunbar 
Lindquist  Center 
for  Measurement 
University  of  Iowa 
Iowa  City,  IA  52242 


Dr.  James  A.  Earles 

Air  Force  Human  Resources  Lab 

Brooks  AFB,  TX  78235 

Dr.  Kent  Eaton 
Army  Research  Institute 
5001  Eisenhower  Avenue 
Alexandria,  VA  22333 

Dr.  John  M.  Eddins 
University  of  Illinois 
252  Engineering  Research 
Laboratory 

103  South  Mathews  Street 
Urbana,  IL  61801 

Dr.  Susan  Embretson 
University  of  Kansas 
Psychology  Department 
426  Fraser 
Lawrence,  KS  66045 

ERIC  Facility-Acquisitions 
4833  Rugby  Avenue 
Bethesda,  MD  20014 

Dr.  Benjamin  A.  Fairbank 
Performance  Metrics,  Inc. 

5825  Callaghan 
Suite  225 

San  Antonio,  TX  78228 

Dr.  Leonard  Feldt 
Lindquist  Center 
for  Measurement 
University  of  Iowa 
Iowa  City,  IA  52242 

Dr.  Richard  L.  Ferguson 
American  College  Testing 
Program 
P.0.  Box  168 
Iowa  City,  IA  52240 
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Dr.  Gerhard  Fischer 
Liebiggasse  5/3 
A  1010  Vienna 
AUSTRIA 

Prof.  Donald  Fitzgerald 
University  of  New  England 
Department  of  Psychology 
Armidale,  New  South  Wales  2351 
AUSTRALIA 

Mr.  Paul  Foley 
Navy  Personnel  R&D  Center 
San  Diego,  CA  92152-6800 

Dr.  Carl  H.  Frederiksen 
McGill  University 
3700  McTavish  Street 
Montreal,  Quebec  H3A  1Y2 
CANADA 

Dr.  Robert  D.  Gibbons 
University  of  Illinois-Chicago 
P.0.  Box  6998 
Chicago,  IL  69680 

Dr.  Janice  Gifford 
University  of  Massachusetts 
School  of  Education 
Amherst,  MA  01003 

Dr.  Robert  Glaser 
Learning  Research 

4  Development  Center 
University  of  Pittsburgh 
3939  O'Hara  Street 
Pittsburgh,  PA  15260 

Dr.  Bert  Green 
Johns  Hopkins  University 
Department  of  Psychology 
Charles  &  34th  Street 
Baltimore,  MD  21218 

Dr.  Ronald  K.  Hambleton 
Prof,  f  Education  &  Psychology 
University  of  Massachusetts 
at  Amherst 
Hills  House 
Amherst,  MA  01003 


Ms.  Rebecca  Hetter 
Navy  Personnel  R&D  Center 
Code  62 

San  Diego,  CA  92152-6800 

Dr.  Paul  W.  Holland 
Educational  Testing  Service 
Rosedale  Road 
Princeton,  NJ  08541 

Prof.  Lutz  F.  Hornke 
Institut  fur  Psychologie 
RWTH  Aachen 
Jaegerstrasse  17/19 
D-5100  Aachen 
WEST  GERMANY 

Dr .  Paul  Horst 
677  G  Street,  #184 
Chula  Vista,  CA  90010 

Mr.  Dick  Hoshaw 
OP-135 

Arlington  Annex 
Room  2834 

Washington,  DC  20350 

Dr.  Lloyd  Humphreys 
University  of  Illinois 
Department  of  Psychology 
603  East  Daniel  Street 
Champaign,  IL  61820 

Dr.  Steven  Hunka 
Department  of  Education 
University  of  Alberta 
Edmonton,  Alberta 
CANADA 

Dr.  Huynh  Huynh 
College  of  Education 
Univ.  of  South  Carolina 
Columbia,  SC  29208 

Dr.  Robert  Jannarone 
Department  of  Psychology 
University  of  South  Carolina 
Columbia,  SC  29208 

Dr.  Douglas  A.  Jones 
P.0.  Box  6640 
Lawrenceville 
NJ  08648 
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Dr.  G.  Gage  Kingsbury 

Portland  Public  Schools 

Research  and  Evaluation  Department 

501  North  Dixon  Street 

P.  0.  Box  3107 

Portland,  OR  97209-3107 

Dr.  William  Koch 
University  of  Texas-Austin 
Measurement  and  Evaluation 
Center 

Austin,  TX  78703 

Dr.  Leonard  Kroeker 
Navy  Personnel  R&D  Center 
San  Diego,  CA  92152-6800 

Dr.  Michael  Levine 
Educational  Psychology 
210  Education  Bldg. 

University  of  Illinois 
Champaign,  IL  61801 

Dr.  Charles  Lewis 
Faculteit  Sociale  Wetenschappen 
Rijksuniversiteit  Groningen 
Oude  Boteringestraat  23 
9712GC  Groningen 
The  NETHERLANDS 

Dr.  Robert  Linn 
College  of  Education 
University  of  Illinois 
Urbana,  IL  61801 

Dr.  Robert  Lockman 
Center  for  Naval  Analysis 
4401  Ford  Avenue 
P.0.  Box  16268 
Alexandria,  VA  22302-0268 

Dr.  Frederic  M.  Lord 
Educational  Testing  Service 
Princeton,  NJ  08541 

Dr.  James  Lumsden 
Department  of  Psychology 
University  of  Western  Australia 
Nedlands  W.A.  6009 
AUSTRALIA 


Dr.  William  L.  Maloy 
Chief  of  Naval  Education 
and  Training 
Naval  Air  Station 
Pensacola,  FL  32508 

Dr.  Gary  Marco 
Stop  3 1 — E 

Educational  Testing  Service 
Princeton,  NJ  08451 

Dr.  Clessen  Martin 
Army  Research  Institute 
5001  Eisenhower  Blvd. 

Alexandria,  VA  22333 

Dr.  James  McBride 
Psychological  Corporation 
c/o  Harcourt,  Brace, 

Javanovich  Inc. 

1250  West  6th  Street 
San  Diego,  CA  92101 

Dr.  Clarence  McCormick 

HQ,  MEPCOM 

MEPCT-P 

2500  Green  Bay  Road 
North  Chicago,  IL  60064 

Mr.  Robert  McKinley 

University  of  Toledo 

Department  of  Educational  Psychology 

Toledo,  OH  43606 

Dr.  Barbara  Means 
Human  Resources 

Research  Organization 
1100  South  Washington 
Alexandria,  VA  223 1 ^ 

Dr.  Robert  Mislevy 
Educational  Testing  Service 
Princeton,  NJ  08541 

Headquarters,  Marine  Corps 
Code  MPI-20 
Washington,  DC  20380 

Dr.  W.  Alan  Nicewander 
Depaetmiby  of  Bfcjrahodiegy 

Oklahoma  City,  OK  73069 
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Dr.  William  E.  Nordbrock 
FMC-ADCO  Box  25 
APO,  NY  09710 

Dr.  Melvin  R.  Novick 
356  Lindquist  Center 
for  Measurement 
University  of  Iowa 
Iowa  City,  IA  52242 

Director,  Manpower  and  Personnel 
Laboratory, 

NPRDC  (Code  06) 

San  Diego,  CA  92152-6800 

Library,  NPRDC 
Code  P201L 

San  Diego,  CA  92152-6800 

Commanding  Officer, 

Naval  Research  Laboratory 
Code  2627 

Washington,  DC  20390 

Dr.  James  Olson 
WICAT,  Inc. 

1875  SouthState  Street 
Orem,  UT  84057 

Office  of  Naval  Research, 

Code  1142PT 
800  N.  Quincy  Street 
Arlington,  VA  22217-5000 
(6  Copies) 

Special  Assistant  for  Marine 
Corps  Matters, 

ONR  Code  00MC 
800  N.  Quincy  St. 

Arlington,  VA  22217-5000 

Dr.  Judith  Orasanu 
Army  Research  Institute 
5001  Eisenhower  Avenue 
Alexandria,  VA  22333 

Wayne  M.  Patience 
American  Council  on  Education 
GED  Testing  Service,  Suite  20 
One  Dupont  Circle,  NW 
Washington,  DC  20036 


Dr.  James  Paulson 
Department  of  Psychology 
Portland  State  University 
P.0.  Box  751 
Portland,  OR  97207 

Dr.  Roger  Pennell 
Air  Force  Human  Resources 
Laboratory 
Lowry  AFB,  CO  80230 

Dr .  Mark  D.  Reckase 
ACT 

P.  0.  Box  168 
Iowa  City,  IA  52243 

Dr.  Malcolm  Ree 
AFHRL/MP 

Brooks  AFB,  TX  78235 

Dr.  Carl  Ross 
CNET-PDCD 
Building  90 

Great  Lakes  NTC,  IL  60088 
Dr.  J.  Ryan 

Department  of  Education 
University  of  South  Carolina 
Columbia,  SC  29208 

Dr.  Fumiko  Samejima 
Department  of  Psychology 
University  of  Tennessee 
Knoxville,  TN  37916 

Mr.  Drew  Sands 

NPRDC  Code  62 

San  Diego,  CA  92152-6800 

Dr.  Robert  Sasmor 
HQDA  DAMA-ARL 
Pentagon,  Room  3E516 
Washington,  DC  20310-0631 
USA 

Dr.  Mary  Schratz 

Navy  Personnel  R&D  Center 

San  Diego,  CA  92152-6800 

Dr.  W.  Steve  Sellman 
OASD(MRAAL) 

2B269  The  Pentagon 
Washington,  DC  20301 
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Dr.  Kazuo  Shigemasu 
7-9-24  Kugenuma-Kaiga 
Fujusawa  251 
JAPAN 

Dr.  William  Sims 
Center  for  Naval  Analysis 
4401  Ford  Avenue 
P.0.  Box  16268 
Alexandria,  VA  22302-0268 

Dr.  H.  Wallace  Sinaiko 
Manpower  Research 

and  Advisory  Services 
Smithsonian  Institution 
801  North  Pitt  Street 
Alexandria,  VA  22314 

Dr.  Richard  Sorensen 
Navy  Personnel  RAD  Center 
San  Diego,  CA  92152-6800 

Dr.  Paul  Speckman 
University  of  Missouri 
Department  of  Statistics 
Columbia,  MO  65201 

Dr.  Martha  Stocking 
Educational  Testing  Service 
Princeton,  NJ  08541 


Mr .  Brad  Sympson 
Navy  Personnel  RAD  Center 
San  Diego,  CA  92152-6800 

Dr.  Kikumi  Tatsuoka 
CERL 

252  Engineering  Research 
Laboratory 
Urbana,  IL  61801 

Dr.  Maurice  Tatsuoka 
220  Education  Bldg 
1310  S.  Sixth  St. 
Champaign,  IL  61820 

Dr.  David  Thissen 
Department  of  Psychology 
University  of  Kansas 
Lawrence,  KS  66044 

Mr .  Gary  Thomasson 
University  of  Illinois 
Educational  Psychology 
Champaign,  IL  61820 

Dr.  Robert  Tsutakawa 
University  of  Missouri 
Department  of  Statistics 
222  Math.  Sciences  Bldg. 
Columbia,  MO  65211 


Dr.  Peter  Stoloff 
Center  for  Naal  Analysis 
200  North  Beauregard  Street 
Alexandria,  VA  22311 

Dr.  William  Stout 
University  of  Illinois 
Department  of  Mathematics 
Urbana,  IL  61801 

Maj.  Bill  Strickland 
|  AF/MPXOA 

I  4E168  Pentagon 

;  Washington,  DC  20330 

|  Dr.  Hariharan  Swaminathan 

,  Laboratory  of  Psychometric  and 

[  Evaluation  Research 

I  School  of  Education 

!  University  of  Massachusetts 

j  Amherst,  MA  01003 


Dr .  Ledyard  Tucker 
University  of  Illinois 
Department  of  Psychology 
603  E.  Daniel  Street 
Champaign,  IL  61820 

Dr.  Vern  W.  Urry 
Personnel  RAD  Center 
Office  of  Personnel  Management 
1900  E.  Street,  NW 
Washington,  DC  20415 

Dr.  David  Vale 
Assessment  Systems  Corp. 

22 33  University  Avenue 
Suite  310 

St.  Paul,  MN  55114 

Dr.  Frank  Vicino 

Navy  Personnel  RAD  Center 

San  Diego,  CA  92152-6800 
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Dr.  Howard  Wainer 
Division  of  Psychological  Studies 
Educational  Testing  Service 
Princeton,  NJ  08541 

Dr .  Ming-Mei  Wang 
Lindquist  Center 
for  Measuremet 
University  of  Iowa 
Iowa  City,  IA  52242 

Dr.  Thomas  A.  Warm 
Coast  Guard  Institute 
P.  0.  Substation  18 
Oklahoma  City,  OK  73169 

Dr.  Brian  Waters 
Program  Manager 
Manpower  Analysis  Program 
HumRRO 

1100  S.  Washington  St. 

Alexandria,  VA  22314 

Dr.  David  J.  Weiss 
N660  Elliott  Hall 
University  of  Minnesota 
75  E.  River  Road 
Minneapolis,  MN  55455 

Dr.  Ronald  A.  Weitzman 
NPS ,  Code  54Wz 
Monterey,  CA  92152-6800 

Major  John  Welsh 
AFHRL/MOAN 

Brooks  AFB,  TX  78223 

Dr.  Rand  R.  Wilcox 
University  of  Southern 
California 

Department  of  Psychology 
Los  Angeles,  CA  90007 

German  Military  Representative 
ATTN:  Wolfgang  Wildegrube 
Streitkraefteamt 
D-5300  Bonn  2 

4000  Brandywine  Street,  NW 
Washinton,  DC  20016 


Dr.  Bruce  Williams 
Department  of  Educational 
Psychology 

University  of  Illinois 
Urbana,  IL  61801 

Dr.  Hilda  Wing 
Army  Research  Institute 
5001  Eisenhower  Ave. 
Alexandria,  VA  22333 

Dr.  Martin  F.  Wiskoff 
Navy  Personnel  R  &  D  Center 
San  Diego,  CA  92152-6800 

Mr.  John  H.  Wolfe 

Navy  Personnel  R&D  Center 

San  Diego,  CA  92152-6800 

Dr .  George  Wong 
Biostatistics  Laboratory 
Memorial  Sloan-Kettering 
Cancer  Center 
1275  York  Avenue 
New  York,  NY  10021 

Dr.  Wendy  Yen 
CTB/McGraw  Hill 
Del  Monte  Research  Park 
Monterey,  CA  93940 


